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Introduction 


Training has been around as an informal process for countless years. Most higher order animals 
require some level of training in hunting, social skills, or other survival related skills to continue 
their existence beyond early infancy. Much of the training is accomplished through imitation, trial 
and error, and good luck. In some ways the essentials of training in aviation have not deviated from 
this original formula a great deal. One of the major changes in aviation and other technical areas is 
that more complex response chains based on a broader base of knowledge are now required. 

“To certify” means many things according to the American Heritage Dictionary of the English 
Language (Morris, 1969). These meanings range from “to guarantee as meeting a standard” to “to 
declare legally insane.” For this discussion, we will use the definition “an action taken by some 
authoritative body that essentially guarantees that the instruction meets some defined standard.” In 
order to make this certification, the responsible body subjects the educational process, training, 
training device, or simulator to some type of examination to determine its adequacy or validity. 


Academic Accreditation 

In the academic community, the certification process is called accreditation. This refers to the 
granting of approval to an institution of learning by an official review board after the school has met 
specific requirements. In the United States, most universities and colleges are accredited through 
regional associations, which are voluntary associations of educational institutions. For example, 
Embry-Riddle Aeronautical University is accredited by the Southern Association of Colleges and 
Schools (SACS), which is the recognized accrediting body in the 11 U.S. Southern states 
(Alabama, Florida, Georgia, Kentucky, Louisiana, Mississippi, North Carolina, South Carolina, 
Tennessee, Texas and Virginia). SACS and other regional associations establish a set of criteria that 
the members must meet. These criteria address areas that are considered important to the effective 
operation of a college or school. In the case of SACS (SACS, 1989) this includes institutional 
purpose, institutional effectiveness, educational program, educational support services, and 
administrative processes. The accreditation process is a personnel intensive procedure involving an 
internal review conducted by the university's faculty followed up by a formal review by a visiting 
team from SACS composed of faculty from other universities and colleges. The process takes over 
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a year and consumes thousands of personnel hours and many thousands of dollars. Generally a 
satisfactory accreditation is valid for a period of ten years before the accreditation must be 
reaffirmed. The reward to the university is that other universities will recognize the credits awarded 
to their students, and also that the university qualifies for many government loan and grant 
programs. Failure to win or retain accreditation can have catastrophic consequences. 

Traditionally this rather complex process has relied upon the expert judgment of subject matter 
experts for both the self-study and the visiting review team. More recently, as the result of pressures 
from state legislatures interested in proof of the value of various college programs, there has been an 
increasing emphasis in the use of more objective, verifiable measures, such as the pre- and post 
testing of students (Did they learn anything?), performance of graduates on licensure examinations 
(Did they learn enough?), to surveys of employers of graduates (Did they learn anything useful?), 
etc. And, as importantly, asking how the institution has used this information to improve its 
programs. As this process of using objective evaluations continues to grow, the accreditation 
process shifts from using construct validity, based upon a systematic review by experts, to using 
empirical validity based upon observable results. 

In addition to the regional accrediting associations, there are many specialized accrediting bodies 
based on specific academic disciplines, such as engineering, business administration, computer 
science, and psychology. Their procedures are similar to the regional associations. While not as 
important as the regional accreditation, the specialized accreditations demonstrate that the programs 
accredited meet the specialized requirements of various professional associations. Since all of these 
accreditations are paid for by the requesting institution, the cost in both time and money is 
significant. 

Since all of the accreditation processes are pass/fail procedures, the outcome is not to guarantee 
academic excellence but to set the level of minimally acceptable academic mediocrity. The primary 
effect is to bring the weaker institutions up to a level of defined acceptability. This assures the 
consumers (the students and their parents) that they will get some reasonable value for their 
investment. However, for our purposes, the use of independent associations to establish and 
regulate accreditation or certification criteria can serve as one type of possible model for the 
certification of training, training devices, or simulators. 


Professional Licensure/Certification 


Another approach to the certification problem can be found in the process of licensure for selected 
professions. The responsibility can be divided between a government regulatory body and a 
professional society. For example, for the licensure of clinical psychologists in the State of Virginia, 
the applicant must have graduated from an American Psychological Association (APA) approved 
graduate program, must have passed an APA national licensing examination, must pass a state 
written examination, and finally a state administered oral examination (Regulations of the Virginia 
Board of Behavioral Science). As in the case of accreditation, the full costs are borne by the 
applicant. Again, the license does not mean that high quality services will be provided by the 
licensed individual. It does mean that sufficient minimum standards have been met so that the 
licensee is not considered to be an undue risk to the public. This joint relationship between a 
professional society and a government regulatory body provides another type of possible model for 
the certification of training, training devices, and simulators. 
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Interestingly, the APA has another level of recognition called the Diplomate. An individual with 
a Ph.D. and appropriate experience can apply for Diplomate status, and after a favorable review of 
credentials and the passing of a special examination, be awarded Diplomate status. This means that 
the association is essentially certifying the individual for private practice of the profession. 
Unfortunately, state licensing procedures do not give special recognition for the Diplomate status: 
the licensing process is the same for an individual with or without the Diplomate. 

This brings up additional issues with respect to certification. The issues are, “who will recognize 
the certification” and “what is its economic value.” With both academic accreditation and 
professional licensing, there is significant economic value for being certified. However, with the 
case of the Diplomate, the certification may have intrinsic value to the recipient of the recognition or 
certification, but have little or no real economic value. 


Training, Training Devices, and Simulation 


Early aircraft simulators tended to look like miniature aircraft with stubby wings and tails. Their 
design gave them face validity. If they looked like airplanes and the instruments and controls 
appeared to be the same, they should be useful in teaching flying skills. Buyers of aircraft 
simulators have consistently had a strong bias toward purchasing devices that looked and acted like 
the real thing without actually becoming airborne. Researchers have tended to follow behind the 
development curve with questions such as: Does the training transfer? How much fidelity is 
enough? What is the cost effectiveness of simulator versus aircraft training? 

A study by Provenmire and Roscoe (1973) used the Link Gat-1 simulators to train pilots to pass 
their final flight check in the Piper Cherokee Aircraft. Student pilots were given either 0, 3, 7, or 1 1 
hours of training in the simulator before continuing their training in the aircraft. The results showed 
that larger amounts of time in the simulator led to larger amounts of time saved in the aircraft; 
however, the amounts of additional flight time saved diminished in the familiar shape of a learning 
curve. These results were important for two main reasons. First, they provided a basis for 
calculating the marginal utility of the simulator. Training in the simulator was cost-effective until the 
Incremental Transfer Effectiveness Ratio dropped below the simulator/aircraft operating cost ratio - 
in this case, about 4 hours in the GAT-1 for training student pilots to pass the final flight check for a 
private pilot's license. Second, the data also indicated that there was an upper limit to the 
transferability of simulator time to improved performance in the aircraft. Beyond a certain level of 
practice, about 8 hours in this case, the students were not showing increased benefits to their aircraft 
performance. This suggests that any within-simulator improvements were simulator peculiar 
without additional transfer value. 

In an extensive review of the use of maintenance simulators for military training, Orlansky and 
String (1981) concluded that student achievement in courses that used maintenance simulators was 
the same as or better than that in comparable courses that used actual equipment trainers. In fact, not 
only was the training as good, it was cheaper. In one case that they cited, the total costs for the same 
student load over a 15 -year period were estimated to be $1.5 million for the simulator and $3.9 
million for the actual equipment trainer; that is, the simulator would cost 38 percent as much to buy 
and use as would the actual equipment trainer. In a subsequent study (Gibson and Orlansky, 
1986), it was noted that student confidence and performance closely paralleled instructor ratings of 
simulator fidelity. They concluded that to make any generalizations about the effectiveness of 
simulator-based training without considering the fidelity of the simulators would be unwarranted. 



140 


Gibson 


Simulators that offer very high fidelity do not represent a serious problem for certification. The 
problem becomes more difficult as training devices depart in various ways from being faithful 
replicas of the aircraft and aircraft systems they represent. While initial students can benefit from a 
variety of relatively low fidelity training devices and simulators, experienced pilots receiving 
refresher training tend to need high fidelity simulators. The FAA Advisory Circular (AC) 120-45A 
specifies the evaluation and qualification requirements for six of a possible seven-level-of-flight- 
training devices. Level 1 is currently reserved and could possibly include PC-based training 
devices. A flight training device is defined by the FAA as: 

a full scale replica of an airplane's instruments, equipment, panels, land controls in 
an open flight deck area or an enclosed airplane cockpit, including the assemblage 
of equipment and programs necessary to represent the airplane in ground and flight 
conditions to the extent of the systems installed in the device does not require a 
force (motion) cueing or visual system; is found to meet the criteria outline in this 
Advisory Circular for a specific flight training device; and in which any flight 
training event or checking event is accomplished. 

PC-based training devices do not meet these criteria, but many are offering some fairly 
impressive approximations. There will be a fairly steady pressure for some type of certification of 
some of these hardware/software combinations for currency or refresher training. 


Trainer Certification and Training Verification 


Assuming that this will eventually happen, there are two problems to be addressed: one is the extent 
that any simulator training transfers and the other is to have a system that will verify the amount of 
flight experience with the training device and the quality of the individuals performance. The 
Provenmire-Roscoe model provides one way to establish the transfer effectiveness and to establish a 
metric for the upper limits of substitution for using the PC-based devices. This may be too costly 
and it may be necessary to assess performance relative to accepted reference simulators. 

This would be similar to the field practice of tests and measures in which paper and pencil 
intelligence tests are generally judged by how well they correlate with the individually administered 
intelligence tests, such as the WAIS. 

The other problem will be that of verification of the actual amounts of practice. Regulators are 
wary of accepting unconfirmed self reporting. Emerging technology may offer some assistance. It is 
currently possible to log onto networked games and play other opponents interactively; and the 
network charges for the time used. In the future, it may be possible to access approved (certified) 
networked software provided that you have the right PC hardware configuration and practice flying. 
The network could keep the necessary records. Another option would be to use a “smart” card 
system that would use the PC and attached “smart” card hardware to provide a record of the training 
hours. Obviously, there would have to be periodic checks in higher fidelity systems to provide the 
training not available on the PC, and check for possible abuses of the system. A pilot who had high 
PC training time but who performed poorly on the “check rides” would lose PC privileges. 
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Conclusions 


Numerous models for training certification exist. All models require either construct validation 
based on expert opinion, or some form of empirical validation that examines the results of the 
training. To be effective, the certification needs to be recognized by the appropriate regulatory 
agencies. 

Techniques exist to assess the training effectiveness of training, training devices, and simulators. 
However, because of the cost and effort required, there is a need to examine the relationship 
between performance on low fidelity devices as a predictor of performance on higher fidelity 
intermediate devices that could be used as reference standards. If PC-based systems win 
certification, there will also be a need to establish a reporting verification system based either on 
network usage or some type of “smart” card. 
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